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ABSTRACT 

PLAN is an ungraded, computer, supported, 
individualized program of education. This paper discusses the 
development of this proqram year by year. The first two years were 
limited, both in curriculum and content pacing. The third year was 
much more com i>rehensive. The fourth year will involve all grades, and 
will have lessons which are behaviorally and discretely defined, 
information about students, and rules for the use of this 
information. A Program of study is developed by checkina state and 
local school system requirements, the students academic historv, and 
the students academic foundations. A core content of the students 
program is then set up. At the secondary level, this program is aimed 
at providing the student with preparation for at least two major lono 
terra goals which have been decided upon by him and his oarents. The 
examples given in the second booklet on usinq student performance 
data show the data upon which revisions of PLAN modules were made as 
well as student performance on the revised materials. By first 
examinino summary results for all items for an obdective, it is 
possible to determine whether or not the general level of performance 
if acceptable, and further whether results indicate general 
misconceptions or errors in the instructional material itself, (<?g) 
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SOME RESULTS OF USING STUDENT PERFORMANCE DATA 
FOR IMPROVEMENT OF INDIVIDUALIZED INSTRUCTIONAL UNITS 1 

Harold F. Rahmlow^ 

American Institutes for Research 

Introduction 

The overall goal of any educational system should be the improvement 
of educational results. In order to improve educational results effectively 
and cfficientlyi there should be a st.vo.ig emphasis on student performance 
data. 

In the paper t "Use of Student Performance Data for the Improvement of 
Individualised Instructional Materials# " read last year to the American 
Psychological Association# the kinds of data used to refine the individualised 
instruction program and illustrations of this procedure were presented 
(Rahmlow# 1969). The purpose of the present paper is to look at some of the 
results of revision work. 

Presented as part of a symposium*. PLAN in Operation -» A Summary 
of Four Years* Experience in the Evolution of an Educational System 
for Divisions 5 and 15, American Psychological Association, Miami 
Beach# Florida# September 5, 1970. 




2 Now associated with The American College of Life Underwriters, Bryn 
Mawr# Pennsylvania. 
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Background 

PLAN is a data-based system of individualized education. Previous 
papers give detailed background on the project (Dunn, Flanagan, Jung, 
Rhetts, Webster, and Wright in Education, 1970; Weisgerber, in press). 

The basic organizational unit of PLAN is the module. A module con- 
sists of a set of behavioral objectives, learning activities for the student, 
and criterion-referenced test items for evaluation. S* .dents are provided 
with documents called teaching-learning units (TLU's) in which, in most 
cases, student objectives are stated along with a specification of the re- 
sources a student might use and the activities he might pursue as he works 
toward the achievement of the stated objective. Figure 1 shows [ortions 
of two PLAN TLU's. When the student feels that he can perform the objec- 
tives of the module, he is given an module test. Student responses are 
submitted for computer processing, and the results of this evaluation are 
reported on an overnight basis to the student and are also stored for future 
reference. 

One of the uses of the stored data is for revision of the instructional 
program. The test data can then be retrieved and summarized by individ- 
ual item and by objective, thus making it possible to evaluate the overall 
level of student performance on an objective as well as to evaluate student 
performance on individual test items. 
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Because the system strives to assist students to master objectives, 
criterion-referenced rather than norm-referenced evaluation is 
appropriate. The distribution of summary scores of items for an objec- 
tive should show a marked skewing in favor of correct responses. 

Likewise, individual items should be at the seventy to ninety percent diffi- 
culty level. 

Example; Test Revision 

If data on student performance indicate that the students arc not 
accomplishing a particular objective, the problem can be attributed cither 
to the student himself or to the system. During initial development, it is 
best to attribute random failure to students and mass failure to the system. 

If a problem is first analyzed for deficiencies In the instructional system, 
it is necessary to look at not only the instructional units but also the evalu- 
ation items. First we will examine a case which illustrates a problem re- 
quiring the revision of the evaluation instruments. 

Let's consider a portion of a science module used by students who are 
in approximately their fifth year of school. The module is entitled, 

"Science of Learning: Forgetting and Relearning. " The objective under 
consideration is, "Tell which variables were controlled in an experiment 
on forgetting and relearning. " Figure 2 shows a portion of a test that in- 
cludes a described experiment U3ed to evaluate the student's ability to 
perform the objective. Figure 3 summarizes student performance on all C 
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five items for the objective. It can be seen that the distribution is more 
the type of a distribution that would be expected from a normative rather 
than from a criterion-referenced evaluation situation. The distribution 
indicated that very few students were really mastering the objective. It 
would seem desirable to investigate student performance on the individual 
items which have been summarized for Figure 3. Figure 4 shows the per- 
cent of students responding correctly to each of the individual items of the 
test. Items 8 and 10 do not seem to be providing a great deal of difficulty, 
whereas items 7, 9, and 11 are most likely the ones that are contributing 
heavily to the poor student performance noted in the summary in Figure 3. 

An examination was made of the teaching-learning unit and of Vhe indi- 
vidual evaluation items themselves. The teaching-learning unit seemed to 
be providing sufficient information and practice for the student to be able 
to acquire the behaviors necessary to perform well on the criterion items. 
One hypothesis advanced for th^ poor student performance was that the 
students were not able to pick out the significant information in the test 
item itself. The test item stimulus was modified as shown in Figure 5 to 
attempt to highlight the essential information. Furthermore, item 11 was 
eliminated from the test because there was not sufficient information pro- 
vided v/ithln the stimulus upon which the student could make a definitive 
response. 

Figure 6 summarizes the student performance on the four items for 
the revised test. As can be seen, the student performance has improved 
considerably, but there still is room for improvement. Again, an analysis 

I 
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was made of the individual items. Figure 7 illustrates this analysis. 

Item 7 is causing the most difficulty and is probably the major contributor 
to the somewhat depressed performance on the total objective. As can be 
seen from the revised test question, the information that is necessary foi 
the student to answer item 7 has been highlighted; therefore, it could be 
concluded that whereas highlighting significant information within the stim- 
ulus improved performance significantly for some of the items, this was not 
the case for item 7. Probably future student improvement would not be 
gained by further refinement of the evaluation instrument but rather by re- 
finement of the instructional procedure itself. 

I'd like to summarize the points made in this section. On the basis of 
student performance data and from an examination of instructional units 
and evaluation documents themselves, it was decided to modify the evalu- 
ation Instrument in an attempt to clarify the criterion situation. The revi- 
sion did seem to produce Improved student performance for some items but 
not for all. On the basis of data collected on students who had used ihe re- 
vised materials, it would be hypothesized that further refinement of the cri- 
terion items would not produce significant results but rather that improved 
results could only be improved by the refinement of the instructional proce- 
dure itself. 

Example; Reorganization of a Module 

Let's now consider an example of a data-based modification of an objec- 
tive as well as a reorganization of instructional materials and the evaluation 
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items. This example is taken from a module in science designed for stu- 
dents who are in approximately their tenth year of school. Figure 8 shows 
the objectives from the original module ami the revised module. 

The original objective was, "Measure the volume of solids and liquids 
in metric units to within one millimeter. " The learning activities asked 
the student to read about the measurement of solids and liquid , It also 
had the students do a number of practical exercises in which they used a 
graduated cylinder and beaker to actually measure different amounts of 
water. Because it was anticipated that students would have to do some 
conversions from one type of unit to another, there are also some activites 
requiring such conversions, for example, from centimeters to millimeters. 

Figure 9 shows the summary across all items of student performance 
data for objective 6685 in the original module. Since the dis*’'’ *»'*** >n of 
correct responses does not indicate a pattern of mastery aud, ' irfhermore, 
since only 12.6 percent of the students answered all four items correctly, 
it appears that students were having difficulty mastering the objective. An 
examination was made of the item responses for each of the four individual 
items. The summary by* items is presented in Figure 10. As can be seen 
for items 12 and 13, thirty-three and thirty- * percent of the students re- 
spectively were selecting the correct . ; mi. Vn examination ( i' both items 
12 and 13 indicated that each was a conversion rather than a direct 

measurement item, whereas items 14 and 15 dealt with actual measurement 
of volume. This lead to an examination of some other items for other 
objectives. It was then discovered that there were a number of items 
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throughout the test which, although related to different types of measure- 
ment, were primarily problems in conversion from one system of meas- 
urement to another. The decision was made to reorganize the module and 
to add an objective dealing specifically with conversion. 

The new objective that was added on conversion was, "Define each of 
the prefixes used to indicate different units of measure in the metric 
system (Mill!-, Centi-, Deci-, and Kilo-). Given a measure of length, 
volume, or weight in any metric unit, convert the measure to any other 
metric unit of length, volume, or weight. " The learning activities of the 
module were then reorganized so that those dealing with the conversion 
were grouped under the new objective and those pertaining primarily to 
the measurement of volume were classfied under the appropriate 
objectives. 

Figure 11 indicates the test results for the revised module. For 
objective 6685, the pattern is better in the revised module than in the 
original module; but, still, there is a relatively small percentage of stu- 
dents (27. 9) who were getting all items correct. On the other hand, for 
objective 6702, the distribution of correct responses is f au</ satisfactory. 

Reviewing the pattern for objective 6685, it would again be useful to 
investigate the particular item responses to see if something can be 
learned from the data. A summary of the individual item responses for 
objective 6685 is given in Figure 12. It can be seen that iterr. number 14 

C 




Page Eight 



is causing a severe amount of difficulty, whereas items 15 and 16 are 
quite satisfactory. Figure 13 shows item 14. Although this item deals 
with measurement of volume, it could more correctly be described as a 
question involving conversion from cubic centimeters to liters. The 
other two items — items 15 and 16 — deal directly with measurement 
skills. On these items the students seem to be doing satisfactorily. We 
could, therefore, conclude that with respect to the objective 6685 itself, 
the students were performing satisfactorily; although it would be good to 
have more items to evaluate student performance. Item 14 is a misplaced 
item dealing with conversion. 

It is interesting to note that item 14, on which the students were having 
severe difficulty, is the same as item 12 on the original test. Looking 
back at the previous test results in figure 10, it can be seen that item 12 
was answered correctly by thirty-three percent of the students initially, 
whereas it was answered correctly by thirty-nine percent of the students 
in the revised module. This is an improvement, but not the type of im- 
provement that would be desired. 

Examining the item responses for item 14 in Figure 12 more closely, 
it can be s >en that although incorrect options A, B, and C each received 
a large number of responses, item C was selected must often. Looking 
back at the item responses for item 12 (Figure 10) from the previous 
year's test, the same is true. Looking at option C, it is s- n that the 
response is 500 cc's, whereas the correct response is 5,000 cc's. It 
would appear that the students are mistakenly assuming that a liter con- 
tains 100 cubic centimeters rather than 1,000. It can be assumed that 
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the students are no;, having difficulty with the conversion between milliliter 
and liter since this is the performance required in item number 11 where 
eighty-one percent of the students are attaining the correct response. The 
real difficulty, then, seems to be the conversion from cubic centimeters 
to milliliters. An examination of the teaching-learning unit indicates that 
there are numerous opportunities for the student to practice conversion 
within the same system — that is, converting centimeters to milliliters, 
or converting milliliters to liters, but there is little emphasis given to 
converting cubic centimeters to milliliters. Thus, the data has indicated 
a potential source of difficulty, and this potential source of difficulty was, 
in fact, found in the materials themselves. Future revision should take 
this problem into account. 

Summary 

The improvement of the educational process is an ongoing activity. 
Attempts at improvement can be completely heuristic or can be based upon 
student performance data. The examples presented have shown the data 
upon which revisions of PLAN modules were made and have shown student 
performance results on the revised materials. 

The data analysis process is not cut and dried, and yet there are some 
general rules which can be followed. By first examining summary results 
for all items for an objective, it is possible to determine whether or not 
the general level of performance is acceptable. A mastery type distribu- 
tion is desirable. Although the initial examination is made on the summary 
of test items, it is possible to know a little about the individual items even 
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from this summary. If the summary is good, then no individual item can 
be too bad; but if the summary indicates difficulty, it is necessary to look 
at the individual items to see where the difficulty is. 

After looking at the items summarized for the objective, it is desir- 
able to examine individual test item response patterns. Clearly, a uniform 
distribution of wrong responses indicates general misconceptions, whereas 
a heavy loading on specific wrong responses signals errors in the instruc- 
tional material or the test itself. Also, a particular alternative may 
clue a wrong response. 

Overall, the improvement of instructional units using student perfor- 
mance data is in the "pre-scientific" stage. Through continual work in 
this area, we hope not only to be able to improve student performance on 
specific objectives but also to discover generalizable paradigms for gene- 
ral improvement. 
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Figure 1 

SAMPLE TEACHING- LEARNING UNITS 



10-251-4 DICTIONAR Y SKILLS AND SIMPLE SENTENCE S (0,00) 



Step 2. Objective: GIVEN A GROUP OF SENTENCES IN WHICH THE SAME UNDERLINED WORD 

(3353) IS USED WITH A DIFFERENT MEANING IN EACH SENTENCE, WRITE THE DEFINITION 
OF THE WORD AS IT IS USED IN EACH SENTENCE. 



USE 



"What's the Word?" Part II -- Suffixes, 
from Reading for Meaning , Houghton Mifflin 
Company, 1966, filmstrip. 

Filmstrip viewer. 



Open Highways , Book 5, "Think-and-Do 
Book." 



DO 

(a) View the filmstrip. 

(b) List suffixes or word endings 
used in the filmstrip. 

Example : 

(1) -ness ’ (6) 

(2) (7) 

(3) (8) 

(4) (9) 

(5) (10) 

(c) Oo the exercise on p. 38. Write 
just the answers. Check your work in 
The Teacher's Edition. 



Open Highways t Book 5, "Think-and-Do 
Book," Teacher's Edition. 



13-315-3 



WHEN AND HOW TRANSFORMS, WORD CLASSES. COMPOSITION 



( 0 , 00 ) 



Step 4. 
(3912) 



Objective: AFTER READING A SELECTION OF LITERATURE, WRITE AN ORIGINAL 

COMPOSITION ABOUT THE PERIOD OF TIME COVERED IN THE SELECTION. 



USE 



DO 



The Roberts English Series, Book 6. 



Spoken English to Accompany the Roberts 
English Series, Grade 6, Harcourt, Brace 
& World, Inc., record set. 



O . 

ERIC 



(a) An author, whose works you will 
read as you continue through school, is 
William Shakespeare, who wrote in 
England about 1600. Many people 
consider him the world's greatest poet. 
Cn p. 107, there is a little song 
about winter from one of the plays he 
wrote. As you listen to a recording of 
this poem on Record 6A, Side One, you 
should remember that at the time it 
was written, winter was a very^ difficult 
time for Jwth 
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ORIGINAL TEST QUESTIONS 



30-264 



MODULE TEST 
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Three of the students Ln Mrs. Bryon's class were doing some experiments. 
Experiment I is described in the paragraph below. Read this paragraph 
carefully before you answer questions 7 through 11. 

EXPERIMENT I 

Each of the three students was given a different list of three-digit numbers 
and allowed one hour in which to memorize his list. The students practiced 
in a quiet area, and when they were finished, took Test I to find out how 
many numbers they had memorized. They all took Test I at the same time, 
in the same room. Six weeks later, they took Test II to see how many of 
the numbers they had remembered. Each student took Test II in a different 
room. None of the students had been allowed to practice between Test I and 
TestH. 

Now, answer questions 7 through 1 1 by using information given in the de- 
scription of Experiment I, above. 



7. The list of numbers memorized by the students was 

A. a controlled variable. 

B. an uncontrolled variable. 

8. The amount of practice time allowed for the students to memorize the 
list was 



A. a controlled variable. 

B. an uncontrolled variable. 



9. The time of day when the students took Test I was 

A. a controlled variable. 

B. an uncontrolled variable. 

10. The time between Test I and Test II was 



A. a controlled variable. 

B. an uncontrolled variable. 



11. The noise in the room where the students took Test II was 

A. a controlled variable. 

B. an uncontrolled variable. 
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SUMMARY ACROSS ALL ITEMS OF STUDENT PERFORMANCE DATA 
FOR ORIGINAL TEST QUESTIONS 

Objective 3102 



Number 
of J^ems 

Correct Frequency Percent 



0 

1 

2 

3 

4 

5 



6 


11. 1 


5 


9. 3 


8 


14. 8 


20 


37. 0 


11 


20. 4 


4 


7. 4 



TOTAL 



54 



100. 0 



MEAN 



2. 69 



STANDARD DEVIATION 



1. 39 
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SUMMARY BY ITEM OF PERCENT OF 
STUDENTS RESPONDING CORRECTLY TO 
ORIGINAL TEST QUESTIONS 



Objective 3102 



Percent 
of Students 

Item Responding 

Number Correctly 



7 

8 
9 

10 

11 



34 

91 

50 
79 

51 
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REVISED TEST QUESTIONS 



30-264 



MODULE TEST 
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Thiee of the students in Mrs. Bryon’s class were doing two experiments. 
The first experiment is described in the paragraph below. Read this para- 
graph carefully before you answer questions 7 through 10. 



EXPERIMENT I 



Each of the three students was given a different list of numbers and allowed 
one hour in which to memorize his list. The students practiced in a quiet 
area and, when they were finished, took Test I to find out how many numbers 
they had memorized. They all took Test I at the same time of day, in the 
s ame room . Six weeks later, they took Test II to see how many of the num- 
bers they had remembered. The students took Test II at the same time but 
in different rooms. None of the students had been allowed to practice be- 
tween Test I and Test II. 

Mow, answer questions 7 through 10 by using information given in the descrip- 
tion of Experiment 7 above. 

7. The list of numbers memorized by the students was 

A. a controlled variable. 

B. an uncontrolled variable. 

8. The amount of practice time allowed for the students to memorize the 
list was 

A. a controlled variable. 

B. an uncontrolled variable. 



9. The time of day when the students took Test I was 



A. a controlled variable. 

B. an uncontrolled variable. 

10. The time between Test I and Test II was 




A. 

B. 



a controlled variable, 
an uncontrolled variable. 
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SUMMARY ACROSS ALL ITEMS OF STUDENT PERFORMANCE DATA 

FOR REVISED TEST QUESTIONS 

Objective 3102 



Number 
of Items 



Correct 


F requency 


Percent 


0 


3 


2. 0 


1 


12 


8. 0 


2 


20 


13. 3 


3 


61 


40. 7 


4 


54 


36. 0 



TOTAL 150 100.0 



MEAN 3.01 

STANDARD DEVIATION . 99 
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SUMMARY BY ITEM OF PERCENT OF 
STUDENTS RESPONDING CORRECTLY TO 
REVISED TEST QUESTIONS 



O bjective 3102 



Item 

Number 



Percent 
of Students 
Responding 
Correctly 



7 

8 
9 

10 



48 

89 

95 

80 
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OBJECTIVES FROM ORIGINAL SCIENCE MODULE 
AND REVISED SCIENCE MODULE 



Objective from Original Module 

668 5 Measure the volume of solids and liquids in metric units to 
within one milliliter. 



Objective from Revised Module 



6685 Measure the volume of solids and liquids in metric units to 
within one milliliter, 

6702 Define each of the prefixes used to indicate different units of 
measure in the metric system (Milli, Centi, Deci, and Kilo). 
Given a measure of length, volume, or weight in any metric 
unit, convert the measure to any other metric unit of length, 
volume, or weight. 
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SUMMARY ACROSS ALL ITEMS OF STUDENT PERFORMANCE DATA 

FOR ORIGINAL MODULE 



Objective 6685 



Numbei 
of Items 

Correct Frequency Percent 



0 

1 

2 

3 

4 



27 


4. 5 


139 


23. 1 


225 


37. 4 


135 


22. 4 


76 


12. 6 



TOTAL 



602 



100. 0 



MEAN 



2. 16 



STANDARD DEVIATION 



1.06 
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SUMMARY BY ITEM OF NUMBER OF STUDENTS 
RESPONDING CORRECTLY TO ORIGINAL TEST QUESTIONS 



Objective 6685 



Item 

Number 


A 


B 


C 


Options 

D 


E 


Omit 


Percent of 
Students 
Responding 
Correctly 


12 


84 


107 


187 


196* 


28 


0 


33 


13 


70 


47 


219* 


161 


103 


2 


36 


14 


396* 


17 


106 


83 


0 


0 


66 


15 


38 


31 


487* 


45 


1 


0 


81 



* Correct response 
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SUMMARY ACROSS ALL ITEMS OF STUDENT PERFORMANCE DATA 

FOR REVISED MODULE 



Objective 6685 



Number 






of Item 8 






Correct 


Frequency 


Percent 


0 


33 


8.4 


1 


84 


21.3 


2 


167 


42.4 


3 


110 


27.9 


TOTAL 


394 


100.0 


MEAN 




1.90 


STANDARD DEVIATION 




.91 



Number 
of Items 
Correct 


Objective 6702 
Frequency 


Percent 


0 


21 


5. 3 


1 


47 


11.9 


2 


61 


15. 5 


3 


103 


26. 1 


4 


162 


41. 2 


TOTAL 


394 


100. 0 


MEAN 




2.P6 


STANDARD DEVIATION 




1.21 




— n»', 
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SUMMARY BY ITEM OF RESPONSES OF STUDENTS 
RESPONDING CORRECTLY TO REVISED TEST QUESTIONS 



Objective 6685 



Item 

Number 



Options 



Percent of 

Students 

Responding 





A 


B 


C 


D 


E 


Omit 


Correctly 


14 


64 


55 


101 


154* 


17 


3 


39 


15 


285* 


24 


33 


52 






72 


16 


23 


30 


309* 


25 


1 


1 


78 



Objective 6702 



Item 

Number 


A 


B 


C 


Options 

D 


E 


Omit 


Percent of 
Students 
Responding 
Correctly 


10 


19 


258* 


27 


18 


71 


1 


65 


11 


26 


23 


318* 


9 


18 




81 


12 


9 


27 


22 


307* 


29 




78 


13 


64 


72 


243* 


13 




2 


62 



♦Correct response 
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Item 14 — Revised Test 



14. How many cubic centimeters (cc) of water could be contained in a 



5 liter 


container? 


A. 


5 cc 


D. 


50 cc 


C. 


500 cc 


D. 


5, 000 cc 


E. 


0. 5 cc 
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The Development of Procedures for the 
Individualization of Educational Programs 

Project PLAN is An attempt to develop an educstionai system which 
will more fully meet the needs of today's youth. Mere specifically, PLAN 
is an ungraded, computer supported, individualized program of education. 

PLAN is learner oriented. Content, rate, and instructional materials are 
tailored to the individual student. Performance requirements are criterion 
referenced and success focused. Tests and materials were developed to 
an 80-80 target, i.e., 80 X content mastery by 80% of the students, on first 
completion of the material. The acronym, PLAN, stands for Program for 
Learning in Accordance with Needs. As the name of the project implies, 
the student's program of studies is the heart of the system. The purpose 
of this psper is to review the development of the PLAN procedures for 
individualizing these programs. 

During the first year of PLAN operation in the schools, because of 
the modest scope of instructional -lterials available (PLAN had been in 
operation as o project only six months when school opened and only grades 
1, 5 and 9 were involved at that time), student planning was perforce quite 
simple. The POS was, in essence, linear. That is, once a student vas 
placed in a particular reading program or course, all students in that course, 
assuming no teacher intervention, were generally recuired to take the same 
instructional materials in the same order. It was the. teachers' responsibility 
to ensure a satisfactory rate of student progress through the materials. 

The only POS related service offered through the computer at that time was 
test scoring and, upon satisfactory student completion of a lesson, the 
assignment of the next lesson in the sequence. 

During the second year of PLAN somewhat more flexibility in curricular 
assignment and somewhat better control over content pacing was achieved. 

During this period the student and his teacher jointly planned what lessons 
the student would take, the order in which they would be taken, and the 
number to be taken. Teacher-pupil planning of this type took place as soon 
as possible after school started and was repeated quarterly throughout the 
school year. Of course, whenever a pupil, or his teacher, thought he vas 





falling behind in his commitment, they could, and frequently did, enter 
into renewed planning to arrive at revised agreements that were more realistic 
and thus, more psychologically binding. A sample of the form used by the 
teacher and the pupil in their planning is given in Figure 1. 

This procedure was considerably more effective in individualizing stu- 
dents' studies than the earlier linear procedure, but it was far from what 
was originally envisioned for PLAN. It xs clearly unrealistic to expect 
the teacher at the beginning of the year to be intimately knowledgeable about 
all of the children in her class, their interests and abilities, their general 
approaches to learning, their long range goals and aspirations, and the like. 

During the third year of operation we were ready for transition to a 
more comprehensive system. We had nine levels of instructional materials 
to draw from, we had fairly extensive testing procedures to use, and we had 
histories on prior PLAN performance for two-thirds of our students. At this 
point we were ready to begin testing procedures wherein the power of the 
computer might be used to generate tentative, or recotnnended, student programs 
of study. The general rationale and characteristics of this procedure were 
reported in some detail at APA last year.* Samples of these early data 
suggested POS's are given in Figure 2. 

This year, for the first time, all 12 grade levels will be represented 
in PLAN. To operate an individualized system such as PLAN, one must hAve a 
relatively large number of lessons which are discretely, and behavlorally, 
defined. Ai.d each of these must have their own assessment procedures. In 
PLAN we have approximately 2,500 teaching-learning units representing 1200 
instructional units or modules. Bach modolu has its own criterion test 
which is taken immediately upon completion of the module and which must be 
mastered before the student proceeds to the next module. 

One also needs considerable Information about students, their interests, 
abilities, achievements, and aspirations. This Spring a two-day battery of 
tests was administered to every student. This testing Included PLAN 



1 James A. Dunn, The Accomodation of Individual Differences in the Develop- 
ment of Personal Programs of Study. Presented at the 1969 APA Convention. 




- 3 - 



Achievement Tests for each of the subject matter areas studied by the 
student and a Developed Abilities Performance teat (patterned after the 
Project TALENT tests which measured such developed abilities as vocabu- 
lary, reading comprehension, arithmetical reasoning, and the like). 

Information on student interests was also collected at that time. In 
addition we also have the student's previous academic history, which has 
been accumulated via module tests throughout the student's tenure in PLAN. 

Finally, one needs a set of decision rules, i.e. specifications and 
guidelines, by which the information about students and materials can be 
interconnected to produce meaningful programs of study. This year's specifi- 
cations for POS consist of over 200 pages of detailed instructions, charts, 
and diagrams; plus an additional 400 pages of detailed tables and module 
coding information describing the characteristics of the 1200 modules in 
the PLAN repertoire. 

Currently, a PLAN student's program of studies (POS) is developed in 
the following manner. First, state and local school system requirements 
are considered and a check of the student's academic history is made to see 
if he has met those requirements. Then his history is reviewed to ascertain 
that he has completed the essential elements of the previous year's work; 
that is, a check is made to see if he has thu necessary academic founda- 
tions to pursue the work that will be expected of him in the coming year. 

Next, the child's achievement test results are considered to see if there 
is anything from last year's material that needs to be reviewed and what, 
if anything, from the coming year's work he may already know. These procedures 
define what might be described as the core content of the student's future 
POS. 

These core requirements are then projected across the time remaining 
for the student to study that particular area. In the early primary grades, 
for example, reading programs generally are of three years' duration. Thus, 
an entering second grader should normally work to finish his basic reading 
instruction in the next two years. It is not essential that he do so; it 
is simply a sore or less reasonable target. A high school student entering 
as a sophomore would have three years to continue his mathematics studies; 
unless of course, he indicated he planned to take mathematics a fewer 
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number of years. 

After the basic, or core requirements are identified and distributed 
across the balance of the time expected to be devoted to the study of that 
content, attention is then turned to determining how much of the requisite 
material should be taken in the immediately ensuing year. If n is the number 
of years remaining to study in an area, then the student is assigned at least 
1/n of the required modules. Typically the core requirements constitute 
much less than a year's worth of study, so attention must shift to the 
assignment of modules to augment these basic core modules. To do this, 
consideration must be given to determining what is a reasonable amount of 
work for the student to cover in a year. 

This is determined by taking into consideration both the student's 
level of developed abilities, as determined by a battery of tests admin- 
istered in the Spring, and also the number of modules the student completed 
the preceding year. 

In the event that a student's quota is not filled by the 1/n requirement 
(an almost guaranteed condition), the POS then begins to assign lessons that 
are considered highly desirable for the student to take. These are lessons 
not considered to be absolutely essential for further academic progress 
but which are nevertheless considered to be very important, basic, content 
for the student to learn. 

If upon completion of assignment of these highly desirable lessons 
the student's quota for the year is still not filled, the remainder of the 
quota is divided evenly between lessons expected to appeal to the special 
interests of the student and to the assignment of required modules, 
from the next higher level. Assignment of modules of this latter type permits 
some measure of student acceleration without sacrificing curriculum enrichment. 

After these lessons, or modules, have been selected, attention is then 
directed toward making recommendations as to which particular instructional 
materials (Teaching-Learning Cnits) the student should use in the study of 
his lessons. The primary factors considered in these recommendations are 
reading difficulty level and amount of social involvement required. 

-* 

( 




Samples of this year's elementary school POS's may be found in 
Figures 3 through 8. 

In the case of secondary school students, the process is somewhat 
more complicated due to the fact that the student's long range educational 
and vocational goals must also be considered. Twelve long range goal 
clusters, or families, which have been identified empirically from Project 
TALENT data are being used in PLAN. An attempt is made to keep student- 
parent planning focused on these general goal clusters rather than on 
specific occupations. This allows us to keep as flexible as possible 
in planning for students. 

From what is known from TALENT, and other sources, about Job mobility 
and the generally poor quality of current student vocational and educa- 
tional planning, PLAN has adopted the philosophy that as many reasonable 
options as possible should be kept open to the student for as long S3 
possible. 

The POS is developed to consider not only the long range goal expressed 
by the student and/or his parents, but also a second "goal" which is the 
result of the best professional Judgment that can be made, given the data 
available for the student. Thus, at the secondary level, the student's 
program of studies aims at providing the student with preparation for at 
least two major long range goal areas. This is felt to be a very important 
characteristic of the secondary level program of studies. Its importance 
is reflected in data obtained in PLAN last year. 

Nearly 2000 PLAN students, and their parents were asked to carefully 
consider their long range educational and vocational plans in order that 
we might accommodate them in their suggested programs of study. 82X indicated 
vocational goals that required a college education, but only 28X indicated 
that they had plans to, or expected to, attend college. (See Table 1.) Had 
educational planning been carried out, as is usually the case, on the expressed 
college-non college expectation of the student (and presumably his parents), 
a large number of families and students would no doubt have been very disappointed 
at some future point in time. 

Similarly, when the expressed vocational goals of students were com- 
pared tu the vocational categories for which they had the ability, it 
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became quite apparent that large numbers of parents and students were 
selecting long range vocational goals well below their ability levels. 

(See Tables 1 and 2.) 

This year FLAN includes a rather extensive set of guidance materials 
aimed at helping students and their parents become much more realistic in 
long range goal formulation and planning. 

In summary then, the major components new this year are: 1) greater 

flexibility in the sequencing of modules; 2) procedures for reviewing 
"essential" academic content that the student had once mastered but had 
apparently forgotten; 3) procedures whereby students might obtain "credit" 
for content which test scores indicated they already knew but which they 
had not formally studied in PLAN; and 4) procedures whereby students and 
parents engage in a series of relatively Intensive exercises aimed at 
helping them arrive at relatively realistic long range educational and 
vocational goals. Thi _eter set of procedures provided for students to 
become much more knowledgeable about the world of work, the world of higher 
education, about their own interests and developed abilities! and lastly 
about the skills and abilities required of various types of long range 
goals. 

As one would expect, PLAN POS's will vary considerably in type and 
amount of content covered, and in the rate and sequerce in which that 
content is covered. 

Uith this approach the concept of a curriculum tends to become 
meaningless. Obviously, from an operational point of view, there are 
as many different curricula as there are discrete programs of study. 

The unit of instruction becomes the individual rather than the class, 
and the unit of credit may be defined in terms of content mastery rather 
than content exposure (e.g., the Carnegie Unit or the semester hour). 

Finally, with regard to some simple operating characteristics, student 
programs of study are run on an IBM Model 50 computer. Over 140,000 units 
of core storage are required Just to store curriculum information about the 

1200 instructional modules *.n the PLAN system. The remaining 8,000 units f 

"i- 
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of storage are required for processing. The processing tine for each 
elementary student's POS (one in each of four subject matter areas: math, 

science, language arts, and social studies) is approximately 10 seconds. 
While this seems like a very brief time, and indeed is very economical, 

10 seconds ot continuous computer processing on a machine such as the 
360-50 represents an extremely large number of decisions for each student. 

In conclusion it should be indicated that, regardless of the nature 
and degree of planning that has gone on, the teacher is the final authority 
in the classroom. The teacher uses the recommended POS as he or she sees 
fit. She may implement the program of study in its entirety; she may make 
minor revisions to it; she may make major revisions to it; or she may even 
ignore it completely and develop an alternative program of studies for the 
student. Whatever her final course of action, she at least will have had 
the best counsel ve could offer. 
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Student-Parent 

Long Range Goal Category 
College Non-College Total 

TABLE 1 

College 

Student-Parent 

Non-College 

Educational Goal 



Total 



X 2 - 12 with 1 d.f. 



538 


121 


659 


28X 


6X 


34X 


1035 


221 


1256 


54X 


12X 


66X 


1573 


342 


1915 


82X 


18X 





TABLE 2 



Data Suggested 
LRG Category 



College 



Non-College 



Total 



Student-Parent 

Long Range Goal Category 
College Non-Coil eg a 



Total 



681 


22 


903 


50X 


16X 


66X 


278 


184 


462 


20X 


13X 


33X 


959 


406 


1365 


70X 


29X 
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X 2 * 33.26 with 1 d.f. .001 
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Student ^jme Subject Area bJr*U Contract Date 



To the Studen t 

This contract Is a tentative agreement between you and one of your PLAN teachers. Its purpose Is to help you, 
your parents, your teacher, and your counselor agree upon plans and decisions made for your future work in one PLAN 
class. If you complete this contract and have helped plan It, you will have derunstrated at least mlnlmim mastery 
of this subject If you do the following: 

1* Complete the modules listed In your Program of Studies during the ttrre period planned; 

2. Her; the check-points listed in your Individualized Schedule In order to complete your Program of Studies 
successful 1y. 

3, Mee* any additional performance requirements listed on the back of this page; and 

A, Score at satisfactory levels on any Survey Tests you take during the time period planned. 

The Information on the back of this page should tell what decisions you have made about the goals you want to 
achieve soon In Project PLAN. These goals will help you manage your learning activities In this class. Therefore, 
Kike sure that you give as complete Irformatlon as you can. Complete Information is also needed for computer proces- 
sing. When you and your teacher have completed your contract, get the necessary signatures below. Decide with your 
teacher whether or not your parents' signature is necessary. 



Make sure that you have one copy of a signed contract for each of your PLAN classes, four teachers might 
want to keep a copy of every contract. 



• 6y November 15, T968 , I plan to have modules completed in this subject area. The number will 

equal about one-quarter of my year's work. 

I agree that this is a tentative contract. My teacher and l may have to revise it If my work during the 
next few weeks shows that my Program of Studies, Individualized Schedule, or other requirements are 
impossible to achieve. If I find that 1 am making much more progress than expected, I shall tell my 
teacher so that we can decide upon more challenging goals and plan a more reasonable schedule. 

Chews' 

// (your signature) 

vS yh/cAcuIl. t ~ 

(Teacner's slgorftVeJ (Parent's signature) (Optional) (Other required signatures) 




I understand that In order to demonstrate minimum mastery of this subject I will need to do the 
following: 

1. complete the modules below with either a "Complete," a "Student Review/ or a "Teacher Certify" 
result by this date: Ur 

(Contract completion date) 



P ROGRAM OF STU01ES 

Fill In the numbers of the modules you plan to complete. 



Module nuflfccr 




Module :<umc>er 




Module number 


x.HS-WI 




3. 73'^3 




s. V3~£7? 






4. <4* ~(iW 




6. 













2. meet this INDIVIDUALIZED SCHEDULE of checkpoints of my progress. If you chose to have checkpoints 
after completing each module, or after a group of modules, or after each step, etc., write this 
specific Information below. If you have selected checkpoint dates, write these here also. Make 
sure you describe compl etel v the kind of schedule you and your teacher chose, j 
. Schedule of checkpoints : txllc foc/c nn V / 

~mIamlu2^= jLJlzjl. far ^/v/Ve ^ ,,s finfc ujLlAljl. 

aaxl mUZ-jAH. ■ 

Person(s) with whom I shall check: 



3. meet these other requirements: 'v, <s ,.(/).> r- 




A. score at satisfactory levels on any Survey Tests I take in this subject during this time period. 
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